Receiver-Initiated Handshaking MAC Based On Traffic Estimation for Underwater Sensor Networks

In underwater sensor networks (UWSNs), the unique characteristics of acoustic channels have posed great challenges for the design of medium access control (MAC) protocols. The long propagation delay problem has been widely explored in recent literature. However, the long preamble problem with acoustic modems revealed in real experiments brings new challenges to underwater MAC design. The overhead of control messages in handshaking-based protocols becomes significant due to the long preamble in underwater acoustic modems. To address this problem, we advocate the receiver-initiated handshaking method with parallel reservation to improve the handshaking efficiency. Despite some existing works along this direction, the data polling problem is still an open issue. Without knowing the status of senders, the receiver faces two challenges for efficient data polling: when to poll data from the sender and how much data to request. In this paper, we propose a traffic estimation-based receiver-initiated MAC (TERI-MAC) to solve this problem with an adaptive approach. Data polling in TERI-MAC depends on an online approximation of traffic distribution. It estimates the energy efficiency and network latency and starts the data request only when the preferred performance can be achieved. TERI-MAC can achieve a stable energy efficiency with arbitrary network traffic patterns. For traffic estimation, we employ a resampling technique to keep a small computation and memory overhead. The performance of TERI-MAC in terms of energy efficiency, channel utilization, and communication latency is verified in simulations. Our results show that, compared with existing receiver-initiated-based underwater MAC protocols, TERI-MAC can achieve higher energy efficiency at the price of a delay penalty. This confirms the strength of TERI-MAC for delay-tolerant applications

Bidirectionally Deformable Motion Modulation For Video-based Human Pose Transfer

Video-based human pose transfer is a video-to-video generation task that animates a plain source human image based on a series of target human poses. Considering the difficulties in transferring highly structural patterns on the garments and discontinuous poses, existing methods often generate unsatisfactory results such as distorted textures and flickering artifacts. To address these issues, we propose a novel Deformable Motion Modulation (DMM) that utilizes geometric kernel offset with adaptive weight modulation to simultaneously perform feature alignment and style transfer. Different from normal style modulation used in style transfer, the proposed modulation mechanism adaptively reconstructs smoothed frames from style codes according to the object shape through an irregular receptive field of view. To enhance the spatio-temporal consistency, we leverage bidirectional propagation to extract the hidden motion information from a warped image sequence generated by noisy poses. The proposed feature propagation significantly enhances the motion prediction ability by forward and backward propagation. Both quantitative and qualitative experimental results demonstrate superiority over the state-of-the-arts in terms of image fidelity and visual continuity. The source code is publicly available at github.com/rocketappslab/bdmm.Comment: ICCV 202

Receiver-Initiated Handshaking MAC Based on Traffic Estimation for Underwater Sensor Networks

In underwater sensor networks (UWSNs), the unique characteristics of acoustic channels have posed great challenges for the design of medium access control (MAC) protocols. The long propagation delay problem has been widely explored in recent literature. However,the long preamble problem with acoustic modems revealed in real experiments brings new challenges to underwater MAC design. The overhead of control messages in handshaking-based protocols becomes significant due to the long preamble in underwater acoustic modems. To address this problem, we advocate the receiver-initiated handshaking method with parallel reservation to improve the handshaking efficiency. Despite some existing works along this direction, the data polling problem is still an open issue. Without knowing the status of senders, the receiver faces two challenges for efficient data polling: when to poll data from the sender and how much data to request. In this paper, we propose a traffic estimation-basedreceiver-initiated MAC(TERI-MAC)to solve this problem with an adaptive approach. Data polling in TERI-MAC depends on an online approximation of traffic distribution. It estimates the energy efficiency and network latency and starts the data request only when the preferred performance can be achieved. TERI-MAC can achieve a stable energy efficiency with arbitrary network traffic patterns. For traffic estimation, we employ a resampling technique to keep a small computation and memory overhead. The performance of TERI-MAC in terms of energy efficiency, channel utilization, and communication latency is verified in simulations. Our results show that, compared with existing receiver-initiated-based underwater MAC protocols, TERI-MAC can achieve higher energy efficiency at the price of a delay penalty. This confirms the strength of TERI-MAC for delay-tolerant applications

SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation

In this paper, we propose a scribble-based video colorization network with temporal aggregation called SVCNet. It can colorize monochrome videos based on different user-given color scribbles. It addresses three common issues in the scribble-based video colorization area: colorization vividness, temporal consistency, and color bleeding. To improve the colorization quality and strengthen the temporal consistency, we adopt two sequential sub-networks in SVCNet for precise colorization and temporal smoothing, respectively. The first stage includes a pyramid feature encoder to incorporate color scribbles with a grayscale frame, and a semantic feature encoder to extract semantics. The second stage finetunes the output from the first stage by aggregating the information of neighboring colorized frames (as short-range connections) and the first colorized frame (as a long-range connection). To alleviate the color bleeding artifacts, we learn video colorization and segmentation simultaneously. Furthermore, we set the majority of operations on a fixed small image resolution and use a Super-resolution Module at the tail of SVCNet to recover original sizes. It allows the SVCNet to fit different image resolutions at the inference. Finally, we evaluate the proposed SVCNet on DAVIS and Videvo benchmarks. The experimental results demonstrate that SVCNet produces both higher-quality and more temporally consistent videos than other well-known video colorization approaches. The codes and models can be found at https://github.com/zhaoyuzhi/SVCNet.Comment: accepted by IEEE Transactions on Image Processing (TIP

VCGAN: Video Colorization with Hybrid Generative Adversarial Network

We propose a hybrid recurrent Video Colorization with Hybrid Generative Adversarial Network (VCGAN), an improved approach to video colorization using end-to-end learning. The VCGAN addresses two prevalent issues in the video colorization domain: Temporal consistency and unification of colorization network and refinement network into a single architecture. To enhance colorization quality and spatiotemporal consistency, the mainstream of generator in VCGAN is assisted by two additional networks, i.e., global feature extractor and placeholder feature extractor, respectively. The global feature extractor encodes the global semantics of grayscale input to enhance colorization quality, whereas the placeholder feature extractor acts as a feedback connection to encode the semantics of the previous colorized frame in order to maintain spatiotemporal consistency. If changing the input for placeholder feature extractor as grayscale input, the hybrid VCGAN also has the potential to perform image colorization. To improve the consistency of far frames, we propose a dense long-term loss that smooths the temporal disparity of every two remote frames. Trained with colorization and temporal losses jointly, VCGAN strikes a good balance between color vividness and video continuity. Experimental results demonstrate that VCGAN produces higher-quality and temporally more consistent colorful videos than existing approaches.Comment: Submitted Major Revision Manuscript of IEEE Transactions on Multimedia (TMM